Atmospheric Chemistry and Physics Comment on “Quantitative performance metrics for stratospheric-resolving

Abstract. This comment focuses on the statistical limi-tations of a model grading, as applied by D. Waugh and V. Eyring (2008) (WE08). The grade g is calculated for a specific diagnostic, which basically relates the difference of means of model and observational data to the standard devia-tion in the observational dataset. We performed Monte Carlo simulations, which show that this method has the potential to lead to large 95%-confidence intervals for the grade. More-over, the difference between two model grades often has to be very large to become statistically significant. Since the con-fidence intervals were not considered in detail for all diag-nostics, the grading in WE08 cannot be interpreted, without further analysis. The results of the statistical tests performed in WE08 agree with our findings. However, most of those tests are based on special cases, which implicitely assum